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Abstract — This paper proposes a verification-based decoding 
approach for reconstruction of a sparse signal with incremental 
sparse measurements. In its first step, the verification-based 
decoding algorithm is employed to reconstruct the signal with 
a fixed number of sparse measurements. Often, it may fail as the 
number of sparse measurements may be not enough, possibly 
due to an underestimate of the signal sparsity. However, we 
observe that even if this first recovery fails, many component 
samples of the sparse signal have been identified. Hence, it is 
natural to further employ incremental measurements tuned to 
the unidentified samples with known locations. This approach 
has been proven very efficiently by extensive simulations. 

Index Terms — Compressed sensing, sparse measurements, low- 
density parity-check (LDPC) codes, verification decoding. 

I. Introduction 

Compressed Sensing (CS) has received much attention in 
recent years |[T]-||5]- For a sparse signal, it essentially does 
not require to sample the signal with the traditional Nyquist 
rates. Basically, one can undersample a sparse signal with a 
much lower rate determined by the sparsity of the signal and 
the idea of which stems from various linear-transform based 
compression approach |fl]. 

Formally, the CS problem in the noiseless setting considers 
the estimation of an unknown and sparse signal vector e £ 
from a vector of linear observations s £ R M , i.e., 

s = He, (1) 

where H £ M. MxN is a fixed matrix known as measurement 
matrix and only a small number (the sparsity index), K « 
N, of elements of e are non-zero. The set containing the 
positions of these elements is known as the support set, defined 
as S = {i £ [1, N] : e, ^ 0}, with cardinality \S\ = K. 

The solution to this system of equations is known to be 
given by the vector that minimizes ||eo||o (Zo-norm) subject 
to s = H ■ eo, which is a non-convex optimization problem. 
In (2, it was established that the vector ei with minimum 
Zi-norm subject to s = H ■ ei coincides with eo whenever 
the measurement matrix satisfies the well-known restricted 
isometry property (RIP) condition. However, it was indicated 
in J6l that Zi-based reconstruction algorithms are non-optimal. 

Recently, there are increased interests in employing sparse 
measurement matrices and message-passing reconstruction al- 



gorithms for CS. With bipartite-graph representations of sparse 
measurements, various message-passing algorithms originally 
developed for decoding sparse-graph codes have been intro- 
duced for reconstruction of a sparse signal |]3], 10, 0, flEJ. 
It was shown that the message-passing decoding algorithms 
can outperform Zi-based theoretical limits. The connections 
between Low-Density Parity-Check (LDPC) codes and CS are 
addressed in detail in (9). 

In this paper, we address the problem of the verification- 
decoding-based reconstruction of a sparse signal from sparse 
measurements. We observed that whenever the verification 
decoder fails, the number of unidentified component sam- 
ples of the sparse signal, however, may be greatly reduced 
compared to the signal length. Therefore, it is possible to 
develop an efficient incremental recovery approach based on 
this observation. 

II. LDPC Coding and Compressed Sensing 

A. LDPC Codes and Syndrome Decoders 

Let C be an (N, K) LDPC code of block length N and 
dimension K, which has a parity-check matrix H = [ft. mj „] 
of M rows, and N columns. For LDPC codes over GF(q), 
the Galois Filed of size q, it means that h m>n £ GF(q). If the 
size of field q is larger than 2, we call them non-binary LDPC 
codes. 

The Tanner graph of an LDPC code is with one-to-one cor- 
respondence with the parity-check matrix. Let G = (V (J C, £) 
be the Tanner graph of a code C with respect to the parity- 
check matrix H — [h mj7l ], where the set of variable nodes V 
represents the codeword bits (or columns of H) and the set of 
check nodes C represents the set of parity-check constraints 
(or rows of H) satisfied by the codeword bits. 

Throughout this paper, we denote the set of variable nodes 
that participate in check m by Af(m) — {n : h m , n ^ 0}- 
Similarly, we denote the set of checks in which variable node 
n participates as A4(n) = {m : h m>n ^ 0}. For each n £ V 
and to £ C, let d v (n) and d c (m) denote degrees of variable 
node 77 and check node m, respectively. 

Now consider that a codeword is transmitted over a g-ary 
symmetrical channel. A codeword c £ F N in C (over the 
field F = GF(q)) should admit the parity-check constraint 



of H ■ c = 0. At the receiver, the received vector c can be 
written as c = c + e, where e £ F N is the additive noise. 
With the received vector c, the decoder tries to estimate c (or 
equivalently e). A syndrome decoder works by first calculating 
the syndrome s = He = H (c + e) = He and then finding 
the most likely error patten e given the syndrome s. For the 
q-ary symmetric channel, the syndrome decoder is a minimum 
Hamming distance decoder. 

For general linear block codes over GF(q), the optimal syn- 
drome decoder often has heavy complexity especially for large 
fields. For LDPC codes, there are various low-complexity but 
suboptimal syndrome decoders, which make a clever use of the 
sparsity of the parity-check matrix H . For a binary LDPC code 
over binary-symmetrical channels, the error patten can often be 
identified by the bit-flipping algorithm. Indeed, the positions to 
be flipped are exactly the error patten induced by the channel, 
i.e., S, if the decoding is successful. When one considers a q- 
ary LDPC code transmitted over a q-ary symmetrical channel, 
this error patten e can be efficiently identified by verification- 
based message-passing decoder especially for large q 171, 
[ 10 1 . Indeed, both bit-flipping and verification decodings can 
find their duals in reconstructing a sparse signal from sparse 
measurements [3|, |10|. 

B. Connection to Compressed Sensing 

Given the observation s G R M , the valid set of signal 
vectors is, in fact, a coset of C with the syndrome s, namely, 

A(s) = {e G R N : H e = s} = e + C, (2) 

where C = {c G M. N : H ■ c = 0} and e is one of coset 
leaders with the minimum Hamming weight and the constraint 
of He = s. In general, the number of coset leaders may not 
be unique. 

The inherent connection between CS and linear codes over 
real numbers has been well exploited in [4|. With a coset 
representation of the valid signal set the connection can 
be further stated as follows. 

Theorem 1: Let H G j^mxn ^ a measuremen t matrix 
and e is the sparse signal of length N. Further, assume that 
the M measurements s = H ■ e and that e has at most K 
nonzero elements, i.e., ||e||o < K. Then the syndrome decoder 
can properly recover the original signal e iff the number of 
coset leaders with the minimum Hamming weight equals 1 . A 
sufficient condition for proper recovering is that the minimum 
Hamming distance of the code C = {c G : H ■ c = 0} 
satisfies d min > 2K + 1. 

Proof: The sufficient condition for proper recovering can 
be deduced as follows. If there is another solution e ^ e, 
we have that ||e||o < K and s = H ■ e. Hence, e — e is a 
codeword of C and ||e — e||o > d m in, which contradicts with 

dmin > 2K + 1. ■ 

III. Compressed Sensing with Incremental Sparse 
Measurements 

A. Verification Decoding with Fixed Sparse Measurements 

For sparse measurements, LDPC matrices have been exten- 
sively employed for sparse measurements 0, |]8], 10. It was 



shown in [9] that parity-check matrices of good LDPC codes 
can be used as provably good sparse measurement matrices 
under basis pursuit. In 0, JT], the verification decoder was 
shown to perform well with sparse measurements. Density 
evolution analysis of verification decoder in compressed sens- 
ing was reported recently in [fTTI . 

As a suboptimal syndrome decoder for linear codes over 
real numbers, the verification decoder with fixed sparse mea- 
surements (i.e., the sparse measurement matrix H is fixed) can 
be described as follows: 

1) If a measurement is zero, then all neighboring variable 
nodes are verified as zero. 

2) If a check node is of degree one, then verify the variable 
node with the value of the measurement. 

3) If two check nodes overlap in a single variable node 
and have the same measurement value, then verify that 
variable node to the value of the measurement. 

4) Remove all verified variable nodes and the edges at- 
tached to them by subtracting out the verified values 
from the measurements. 

5) Repeat steps 1-4 until decoding succeeds or makes no 
further progress. 

Now, let us consider what happens whenever the verification 
decoder cannot make further progress (or simply converge). 
When the verification decoder converges after several itera- 
tions, the variable nodes V can be partitioned into two disjoint 
sets, namely, the set of identified nodes Vj and the set of 
unidentified nodes Vu with V = Vi U Vu . If the decoder fails 
to recover the signal, it is of high probability that \Vjj\ > 0. 

B. Verification Decoding with Incremental Sparse Measure- 
ments 

In practice, when the sparsity of the signal can be well 
estimated, it is up to the number of sparse measurements 
for proper recovering. However, when the sparsity is under- 
estimated, the verification decoder may fails and more mea- 
surements are required. In general, one can employ incremen- 
tal strategy for sparse measurements, which requires the design 
of rate-compatible sparse matrices just like one encounters 
in incremental redundancy hybrid automatic repeat request 
(ARQ) schemes. With verification decoding, one may employ 
the binary parity-check matrices developed for rate-compatible 
LDPC codes fT2l . 

In this paper, we, however, did not consider this approach 
but resort to a much simpler approach. In simulations of veri- 
fication decoding, it was noted that the number of unidentified 
variables after the verification decoder converges is often very 
limited compared to the signal length N, i.e., \Vjj\ "C N. 
Hence, it is now natural to employ an incremental measure- 
ment strategy with additional measurements tuned directly to 
the samples with determined locations in Vu- 

Here, we propose a two-step approach for the verification 
decoding with incremental measurements. In its first step, a 
sparse measurement matrix H = [h m , n ] is employed with M 
measurements. If the decoder fails, possibly due to an under- 



estimation of the sparsity, it initiates an incremental approach 
with direct sampling. 



proposed in [ 8 1 for further improving the performance of the 
verification decoding is not adopted. 



1) 



2) 



Algorithm 1 Verification Decoding with Incremental Mea- 
surements 



- Definition: 
k : iteration counter; 

Ko : the threshold for the number of iterations; 
I : incremental measurement couter; 
lm ■ the maximum number of measurements. 

- Initialize: 

set k = 1, I = 0, V/ = 0. 

If s m = Y^n h m,n.e n = for some m G [1,M], its 
neighboring variable nodes are all identified as zeros, 
i.e, e„ = 0,Vn G Af(m). Update the set of identified 
variable nodes as V/ <— V/ (J A/"(m). 
If d c {m) = 1 for some m G [1,-^] and assume 
that 7V(rn) = {n}, its neighboring single variable 
node can be identified as e„ = s n . Update V/ as 
Vj «- Vi [jM{m). 

3) For each n S [l,iV], search over its neighboring check 
nodes to check if there exists two different check nodes 
mi, mi G M.{n),mx ^ m-i with equal but nonzero 
measurements s mi = s m2 ^ 0. If yes, the variable node 
n is identified as e„ = s mi and other variable nodes are 
identified as e n < = 0,Vn' G N(mi)\JN(m 2 ) - {n}. 
Update V/ as V/ <- V/ U-^(m x ) (J Af(m 2 ). 

4) Remove all identified variable nodes and the edges 
attached to them by subtracting out the identified values 
from the measurements. 

5) If k > Ko, go to Step 6). Otherwise, go to Step 7). 

6) If / < Lm, locate the check node m of minimum degree 
(choose any one of them if there are multiple such check 
nodes). Randomly choose an unidentified variable node 
n neighboring to m, i.e., n G Vjj H Af(m) and directly 
sample the signal at this location. Update V/ as V/ <— 
V/lJ{n} and increase Z by 1 . 

7) Increase k by 1, and repeat Steps l)-6) until either all the 
variable nodes are identified or the maximum number of 
iterations is reached. 

IV. Simulation Results 

In this section, we provide the simulation results for the pro- 
posed verification decoding with incremental measurements. 
For source sparse signals, we consider the Gaussian sparse 
case where the entries of the signal are either or a Gaussian 
random variable with zero mean and unit variance. 

We adopt the binary MacKay-Neal LDPC matrices 1(1311 
as sparse measurement matrices. Two LDPC code matrices 
are employed, one of which is with the size of M x N = 
738 x 4095, and the other is with the size of M X N = 
2131 x 16383. Here, M can be seen as the number of initial 
sparse measurements. It should be noted that this matrix is 
constructed without 4-cycles. Hence, the enhanced mechanism 




Fig. 1. \Vjj\/K versus K conditioned on the decoding failure 
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Fig. 2. Probability of decoding failure against the number of additional 
measurements (N = 4095, M = 738). 



A. Distribution of \Vu\ Conditioned on the Decoding Failure 

The size of unidentified variable nodes is investigated 
whenever the verification decoder fails to recover the original 
sparse signal. Here, we focus on the measurement matrix of 
M x N = 738 x 4095. 

The decoding failure occurs more frequently when the num- 
ber of non-zeros elements K increases. Indeed, the probability 
of decoding failure is P f = 1.47e - 006 when K = 250 
while it soon increases to Pf = 0.9 when K = 300. In Fig. 
[T] we show the value of |V[/|/-K^ versus K conditioned on 
the decoding failure, which is averaged over 1000 decoding 
failures. As shown, much information about the sparse signal 
can be well retrieved even when the decoder fails to recover the 
original sparse signal. In most cases, the size of unidentified 
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Fig. 3. Probability of decoding failure against the number of additional 
measurements (N = 16383, M = 2131, K = 800). 



variable nodes is greatly reduced compared to the length of 
the source signal N. Hence, it is of helpful for the decoder 
to further use this information when additional measurements 
are allowed to recover the signal. 

B. Decoding Performance Under Incremental Measurements 

If the number of sparse measurements keeps fixed, the 
probability of successful recovering the signal decreases as the 
number of non-zero elements K increases. In this subsection, 
we investigates the number of additional measurements for 
reducing the probability of decoding failure when the sparsity 
of the signal keeps fixed. 

As shown in Fig. [2 the probability of failure decreases 
rapidly with the number of incremental measurements. For 
K = 280, the additional 10 measurements can reduce Pf 
from about lO^ 1 to below 10~ 4 . For K — 300, it clearly 
requires more measurements. Indeed, about 10 additional 
measurements are required for Pt reaching about 10 _1 . Once 
the probability of failure Pf reaches around 10 _1 , the further 
employment of 10 measurements can again reduce Pf to 
around 10~ 4 . Therefore, one could deduce that the additional 
one measurement can reduce Pf to half of it. Let the number 
of additional measurements be L, the probability of failure 
approximately obeys the following empirical estimation rule 



Pf(L) = P f (0)a 



(3) 



when Pf(0) is around 10 _1 . The parameter a seems to be 
related to both the source sparse signal and the underlying 
sparse measurement matrix. As shown in Fig. [2] the empirical 
estimation with a = 0.5 is plotted. The empirical estimation 
is also plotted with a = 0.65 in Fig. [3] for the sparse matrix 
of size M x N = 2131 x 16383 and the sparsity of the signal 
is K = 800. As shown, the empirical estimation coincides 
well with the simulation results. The exponential decay of the 
probability of decoding failure validates the efficiency of the 
proposed algorithm with incremental measurements. 



V. Conclusion 

We have proposed a verification-based reconstruction algo- 
rithm with incremental sparse measurements. With incremental 
measurements tuned to the sparse signal at the unidentified 
positions, it has been proven very efficient for reducing the 
probability of decoding failure. With an additional direct- 
sampling approach, the implementation complexity is very 
low. 
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